Goto

Collaborating Authors

 Professional


A Human-Centered Approach to Identifying Promises, Risks, & Challenges of Text-to-Image Generative AI in Radiology

Morrison, Katelyn, Mathur, Arpit, Bradshaw, Aidan, Wartmann, Tom, Lundi, Steven, Zandifar, Afrooz, Dai, Weichang, Batmanghelich, Kayhan, Eslami, Motahhare, Perer, Adam

arXiv.org Artificial Intelligence

As text-to-image generative models rapidly improve, AI researchers are making significant advances in developing domain-specific models capable of generating complex medical imagery from text prompts. Despite this, these technical advancements have overlooked whether and how medical professionals would benefit from and use text-to-image generative AI (GenAI) in practice. By developing domain-specific GenAI without involving stakeholders, we risk the potential of building models that are either not useful or even more harmful than helpful. In this paper, we adopt a human-centered approach to responsible model development by involving stakeholders in evaluating and reflecting on the promises, risks, and challenges of a novel text-to-CT Scan GenAI model. Through exploratory model prompting activities, we uncover the perspectives of medical students, radiology trainees, and radiologists on the role that text-to-CT Scan GenAI can play across medical education, training, and practice. This human-centered approach additionally enabled us to surface technical challenges and domain-specific risks of generating synthetic medical images. We conclude by reflecting on the implications of medical text-to-image GenAI.


Benchmarking Generative AI for Scoring Medical Student Interviews in Objective Structured Clinical Examinations (OSCEs)

Geathers, Jadon, Hicke, Yann, Chan, Colleen, Rajashekar, Niroop, Sewell, Justin, Cornes, Susannah, Kizilcec, Rene, Shung, Dennis

arXiv.org Artificial Intelligence

Introduction. Objective Structured Clinical Examinations (OSCEs) are widely used to assess medical students' communication skills, but scoring interview-based assessments is time-consuming and potentially subject to human bias. This study explored the potential of large language models (LLMs) to automate OSCE evaluations using the Master Interview Rating Scale (MIRS). Methods. We compared the performance of four state-of-the-art LLMs (GPT-4o, Claude 3.5, Llama 3.1, and Gemini 1.5 Pro) in evaluating OSCE transcripts across all 28 items of the MIRS under the conditions of zero-shot, chain-of-thought (CoT), few-shot, and multi-step prompting. The models were benchmarked against a dataset of 10 OSCE cases with 174 expert consensus scores available. Model performance was measured using three accuracy metrics (exact, off-by-one, thresholded). Results. Averaging across all MIRS items and OSCE cases, LLMs performed with low exact accuracy (0.27 to 0.44), and moderate to high off-by-one accuracy (0.67 to 0.87) and thresholded accuracy (0.75 to 0.88). A zero temperature parameter ensured high intra-rater reliability ($\alpha = 0.98$ for GPT-4o). CoT, few-shot, and multi-step techniques proved valuable when tailored to specific assessment items. The performance was consistent across MIRS items independent of encounter phases and communication domains. Conclusion. We demonstrated the feasibility of AI-assisted OSCE evaluation and provided benchmarking of multiple LLMs across multiple prompt techniques. Our work provides a baseline performance assessment for LLMs that lays a foundation for future research in automated assessment of clinical communication skills.


Promoting AI Competencies for Medical Students: A Scoping Review on Frameworks, Programs, and Tools

Ma, Yingbo, Song, Yukyeong, Balch, Jeremy A., Ren, Yuanfang, Vellanki, Divya, Hu, Zhenhong, Brennan, Meghan, Kolla, Suraj, Guan, Ziyuan, Armfield, Brooke, Ozrazgat-Baslanti, Tezcan, Rashidi, Parisa, Loftus, Tyler J., Bihorac, Azra, Shickel, Benjamin

arXiv.org Artificial Intelligence

As more clinical workflows continue to be augmented by artificial intelligence (AI), AI literacy among physicians will become a critical requirement for ensuring safe and ethical AI-enabled patient care. Despite the evolving importance of AI in healthcare, the extent to which it has been adopted into traditional and often-overloaded medical curricula is currently unknown. In a scoping review of 1,699 articles published between January 2016 and June 2024, we identified 18 studies which propose guiding frameworks, and 11 studies documenting real-world instruction, centered around the integration of AI into medical education. We found that comprehensive guidelines will require greater clinical relevance and personalization to suit medical student interests and career trajectories. Current efforts highlight discrepancies in the teaching guidelines, emphasizing AI evaluation and ethics over technical topics such as data science and coding. Additionally, we identified several challenges associated with integrating AI training into the medical education program, including a lack of guidelines to define medical students AI literacy, a perceived lack of proven clinical value, and a scarcity of qualified instructors. With this knowledge, we propose an AI literacy framework to define competencies for medical students. To prioritize relevant and personalized AI education, we categorize literacy into four dimensions: Foundational, Practical, Experimental, and Ethical, with tailored learning objectives to the pre-clinical, clinical, and clinical research stages of medical education. This review provides a road map for developing practical and relevant education strategies for building an AI-competent healthcare workforce.


UT Southwestern teaches med students that 'gender is independent of physical structure, chromosomes, or genes'

FOX News

Nineteen protesters were arrested at the Kentucky Capitol on Wednesday amid a protest against a measure that would ban certain gender care for minors. Documents obtained by Fox News Digital show that University of Texas Southwestern medical students are being taught that gender is independent of physical structure. Fox News Digital obtained the documents via a FOIA request from Do No Harm, a national association of medical professionals that combats "woke" activism in the healthcare system. According to the University of Texas Southwestern Medical Center's Human Structure curriculum, they "explicitly acknowledge the differentiation between the terms sex and gender." RACHEL LEVINE SAYS CHANGING KIDS' GENDERS WILL SOON BE FULLY EMBRACED: 'WHEELS WILL TURN ON THIS' "The latter is a psychological, social, and cultural construct, including self-identification. Gender is independent of physical structure, chromosomes, or genes," curriculum materials read.


My 2-year journey into deep learning as a medical student -- Part II: Courses

#artificialintelligence

Deep learning and machine learning courses that I've taken along the way in learning deep learning. It's time to introduce the courses that I've used along this way that helped me get started and grow in the field. You should also keep in mind that there are probably many more and newer courses out there as the community keeps providing interesting educational material every day. So, keep on searching too. This fact aside, I believe the following list introduces high quality courses for many fields that most of you will be okay to start with and learn lots of new things from.


Readying Medical Students for Medical AI: The Need to Embed AI Ethics Education

Quinn, Thomas P, Coghlan, Simon

arXiv.org Artificial Intelligence

Medical students will almost inevitably encounter powerful medical AI systems early in their careers. Yet, contemporary medical education does not adequately equip students with the basic clinical proficiency in medical AI needed to use these tools safely and effectively. Education reform is urgently needed, but not easily implemented, largely due to an already jam-packed medical curricula. In this article, we propose an education reform framework as an effective and efficient solution, which we call the Embedded AI Ethics Education Framework. Unlike other calls for education reform to accommodate AI teaching that are more radical in scope, our framework is modest and incremental. It leverages existing bioethics or medical ethics curricula to develop and deliver content on the ethical issues associated with medical AI, especially the harms of technology misuse, disuse, and abuse that affect the risk-benefit analyses at the heart of healthcare. In doing so, the framework provides a simple tool for going beyond the "What?" and the "Why?" of medical AI ethics education, to answer the "How?", giving universities, course directors, and/or professors a broad road-map for equipping their students with the necessary clinical proficiency in medical AI.


Preparing Medical Students for the Impact of Artificial Intelligence

#artificialintelligence

Today, emerging technologies of the such as artificial intelligence, gene editing, nanotechnology, and the blockchain are being explored as ways to fundamentally "disrupt" medicine and healthcare. Despite the promises of such technologies, implementing this kind of disruption has presented countless unintended challenges. Given, first and foremost, the Hippocratic duties of healthcare providers to'do no harm', it is essential that the role of these emerging technologies in medicine is carefully scrutinized by practitioners that understand and can think critically about them. Artificial intelligence (AI) can be broadly defined as the ability for a machine to perform human-like tasks after learning from experience. AI is poised to introduce significant changes to medicine and healthcare.


Does screen time really affect medical students' surgery skills?

BBC News

Medical students are losing the dexterity to stitch up their patients because they are spending too much time in front of screens. That's the claim from one professor of surgery who says trainee doctors are "less competent", compared to their older colleagues, at using their hands. Professor Roger Kneebone, from Imperial College London, says young people have so little experience of craft skills that they struggle with practical tasks. It's a worrying thought - but how true is it? Does screen time impact dexterity?


Med Students Are Getting Terrible Training in Robotic Surgery

WIRED

If you think your on-the-job training was tough, imagine what life is like for newbie surgeons. Under the supervision of a veteran doctor, known as an attending, trainees help operate on a real live human, who might have a spouse and kids--and, if something goes awry, a very angry lawyer. Now add to the mix the da Vinci robotic surgery system, which operators control from across the room, precisely guiding instruments from a specially-designed console. In traditional surgery, the resident gets hands-on action, holding back tissue, for instance. Robotic systems might have two control consoles, but attendings rarely grant residents simultaneous control.